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1.  Introduction 

The  equality  presented  in  this  paper  is  extremely  simple,  but  has  proven 
useful  on  a  number  of  occasions  in  deriving  certain  transition  probabilities  where 
other  approaches  such  as  the  Laplace  transform  or  the  generating  function  be¬ 
come  untidy  (see,  for  example,  [7]).  The  equality  is  not  completely  unknown ; 
it  has  appeared,  in  a  slightly  different  form,  in  a  two  dimensional  process  ([2], 
p.  102);  and  it  is  obvious  for  the  simple  Poisson  process.  The  purpose  of  this 
paper  is  to  state  it  in  a  general  form  and  to  demonstrate  its  validity  and  useful¬ 
ness  with  a  number  of  examples. 


2.  The  equality 


Let  (XCi);  t  e  T}  he  &  time  dependent  Markov  process  defined  over  the  in¬ 
terval  T:  [0,  oo).  For  each  t  e  T,  the  random  variable  X{t)  assumes  nonnegative 
integer  values  with  the  transition  probability 

(1)  Pikik  t)  =  Pr{X{t)  =  k\X{U)  =  i}, 

0  ^  to  ^  t  <  ^ ,  i  S  k;  i,  k  =  0,  1,  •  •  •  . 
Our  discussion  is  related  only  to  nondecreasing  processes  where  the  value  of  X{t) 
is  increased  by  the  occurrence  of  an  event  (for  example,  the  pure  birth  process), 
or  the  nonincreasing  processes  (for  example,  the  pure  death  process).  The  equal¬ 
ity  will  be  presented  only  for  the  former  cases.  However,  an  example  of  the  pure 
death  process  will  be  given  in  Section  3. 

For  each  i,  we  assume  the  existence  of  a  continuous  function  Xi(T)  such  that 


(2) 


<  =  T 


'X<(t) 

■  -Ur) 
.0 


for;  =  i  +  1, 
forj  =  i, 
otherwise. 


It  follows  that  the  transition  probabilities  in  (1)  satisfy  the  forward  differential 
equations 


i  ^  k;  i,  k  =  0,  1,  •  •  •  . 
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The  first  equation  in  (3)  has  the  solution 

(4)  PiiiU,  t)  =  Oiito,  t) 

where  gi(to,  t)  denotes  exp  while  the  second  equations  give  the 

recursive  relationship 

(5)  Pikitoft)  =  [  Pi^k-iita,  tk-i)\k-i(tk-i)Pkk{tk-i,  t)  dtk-i. 

jt« 

The  probability  Pi,k-i{to,  4-i)  may  be  written  in  the  form  of  (5): 

(6)  Pi,k-l{tQf  tk-l)  =  f  Pi,k-2(to,  tk-2)^k-2(tk~2)Pk-l,k-l(tk-2y  tk-l)  dti-2. 

Jto 

Substituting  (6)  in  (5)  yields 

(7)  Pi,k{tQ,  =  [  [  Pi.k-2(t,  tk-2)  n  \({tt)Pt+l,t+l{t(,  tt+i)  dtt, 

Jtc  Jta  t=k-2 

or,  upon  integration  with  respect  to  4-i, 

(8)  Pi.kiUyt)  =  Pi,k-2(to,  tk-2)Xk-2itk-2)Pk-l,k{tk-2,  t)  dtk-2. 

Equation  (8)  indicates  a  sequence  of  transitions  taking  place  in  the  interval 
{to,  —  2 in  {U,  tk-2),  k  —  2—^k  —  lin  (4-2,  4-2  +  dtk-2),  and k  —  1—^k 

in  (4-2  +  dtk-2,  t),  for  4  ^  4-2  ^  t. 

Equation  (7)  can  be  extended  by  repeated  substitutions  of  (5)  in  (7)  beginning 
with  k  =  k  —  2.  Consequently,  we  arrive  at  a  formula, 

(9)  PM,  t)  =  /'“■■  •  •  •  PM  U)  n'uti)Pi+i.M{ii,  «<+i)  df,, 

Jk  Jk  Jk 

which  shows  the  occurrence  of  each  transition  from  /  to  +  1  in  (4,  4  +  dti). 
Now  the  transitions  can  be  regrouped  in  any  meaningful  way  one  wishes  by 
integrating  the  right  side  of  (9).  When  a  particular  transition,  j  +  1,  is  of 
concern,  we  integrate  the  right  side  of  (9)  with  respect  to  4,  for  i  ^  ^  <  j  and 
j  <  t  ^kto  obtain  the  equality.  Letj  be  an  arbitrary  but  fixed  integer,  for  i  <  k, 

then 

(10)  Pik{to,  t)  =  r  Pij{to,  T)Xy(r)Py+i.fc(T,  t)  dr. 

Jto 

Clearly,  (5)  is  a  special  case  of  the  equality  (10). 

An  equality  for  the  continuous  case  also  can  be  derived,  where  we  will  be 
dealing  with  the  density  function  /x*(4,  t)  instead  of  the  probability  Pik{to,  t). 

A  similarity  between  the  present  equality  and  the  Chapman-Kolmogorov 
equation  should  be  noted.  The  transition  probability  in  (1)  involves  the  param¬ 
eter  t  and  the  values  of  the  random  variable  X{t)  (or  the  states  of  the  system). 
The  Chapman-Kolmogorov  equation, 

(11)  Pikito,  0  =  1:  PM,  T)Pjk{T,  t), 
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is  related  to  a  fixed  but  arbitrary  time  t  and  varying  states  j  at  r;  whereas  in 
equality  (10)  a  states  is  fixed  and  the  integral  is  taken  over  the  values  of  r.  This, 
however,  is  the  extent  of  the  similarity.  In  equality  (10),  we  require  a  transition 
from  j  to  j  +  1  to  take  place  in  (t,  r  +  dr)  for  which  a  probability  \j{T)  dr  is 
included,  while  in  the  Chapman-Kolmcgorov  equation  there  is  no  such  require¬ 
ment.  This  additional  factor  nullifies  a  complete  analogy  between  the  two 
equalities. 


3.  Examples 

3.1.  The  Poisson  process.  In  the  nonhomogeneous  case,  the  intensity  Xi(T)  = 
X(r)  is  a  function  of  time;  the  transition  probability  is  given  by 


(12) 


PikiU,  t)  = 


exp{-/Jx({)d{}[/^(f)d{]‘  • 


i  <  k. 


{k-i)\ 

According  to  equality  (10),  we  should  have 

=  II  [(f^  (f  (/' 

where  g{to,  t)  denotes  exp  X(^)  and  fif(r,  t)  denotes  exp  {  — ^  X(^) 
To  verify  (13),  we  introduce  a  function 


(14) 


(  X(?)  d^ 
=  Tt - » 

L  <« 


X(t)  dr 


dOir)  =  - > 

and  rewrite  the  right  side  of  (13)  as 

(■»  (J‘ «f) «r7.' «'»"!■  - •I'a-" «'> 

-m, 

and  recover  the  left  side  of  (13). 

When  X(r)  =  X,  and  fo  =  0,  (13)  becomes 

/-.^N  e-’^‘(X0*“‘  r  6-^(‘-^>[X(i  -  t)]*-'-i  ^ 

“  Jo  U  -  i)!  (i-J-D! 

3.2.  The  Yule  process.  When  the  intensity  Xi(T)  =  ^X(T),  the  differential 
equations  in  (3)  define  the  Yule  process, 

=  (k  -  1)! 


(17) 
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To  verify  equality  (10),  we  need  to  show  that 


(18) 


{k  -  1)! 


{k  -  i)!(i-  1)! 

(j 


=  / 


(g{to,  t)y{i  -  giu,  t)y- 
1)1 


(i-^)!(i-i)! 


(giio,  r))*(l  —  g{to,  T)y  V‘X(r) 


(fc  -  j  -  l)!j!  ~  <))*^^*  dr, 


where  g{to,  r)  denotes  exp  {-/jX({)d£}-Let 

1  —  g(r,  t) 


(19) 
so  that 


dir)  = 


1  -  ^(<0,  0 


(20)  <M(t) - g(T,  oy’’)  ^  and  1  -  9(t)  = 

1  -  g(*o,  t)  '  '  1-  j((o,  t) 

Substituting  (19)  and  (20)  in  the  right  side  of  (18)  and  simplifying  yields 

(21)  (j  _  l)!(l  -y  -  1)! 


(/fc  -  1)! 


/: 


[1  —  ^(t)]^~*[0(t)]*~*  ^  d0(T) 


ig{io,  0)*(i  -  ^(<0,  <))*■*, 


(A;  -  t)!(»  -  1)! 

which  is  equal  to  the  left  side  of  (18),  proving  (18). 

3.3.  The  pure  birth  process.  To  verify  equality  (10)  for  the  present  case,  we 
need  the  following  lemma. 

Lemma  1.  Whatever  may  he  distinct  real  numbers,  Xi,  X2,  •  •  •  ,  X„, 


(22) 


1 


+ 


n  (Xi 

7=2 


\y) 


+  ;r=r 


1 


=  0. 


n  (Xn  -  x^) 

7=1 


Several  proofs  of  the  lemma  are  given  in  Chiang  [2],  p.  126;  we  do  not  present 
the  details  here. 

The  pure  birth  process  is  a  different  form  from  the  Yule  process  in  that  the 
intensity  function  X,(r)  =  X*  is  a  function  of  i,  but  is  assumed  to  be  independent 
of  time  T.  When  X<  are  distinct  for  different  i,  then  the  differential  equation  (3) 
has  the  solution  (see  Feller  [5],  p.  449,  and  [2],  p.  51), 


(23)  Pik{to,t)  =  (-l)"-‘X,X.+i 
Similarly, 


(24) 


Pii(.to,T)  =  (-1)»-‘X,X<+1 


^  exp  {-Kit  —  to)} 

K-i  Z.  1 - 

n  (x„  -  X7) 

7  =  t,7^a 

^  4.  exp  {— X„(t  —  to)} 

K-l  Z-  y - 

n  (x«  -  X7) 

y=i,y9^a 
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(25)  Py+...(r,()  =  t  ’•)>. 

*'■'+*  n  (Xj  -  Xi) 

5=j+l.S9iff 

When  (24)  and  (25)  are  substituted  in  (10),  we  find  on  the  right  side  of  (10), 


(26)  f  Pij{to,  T)X,P,+i,t(T,  t)  dr 
Jto 


j  k  Texp  {-X«(t  -  ^o)}  exp  {-X/j(<  -  r)}  dr 

=  (-1)^--1X<X<+1  •  •  •  Xfc_i  Z  L  - — -i - 1 - » 

jj  (X.-X,)  n  (x,-x,) 

y=i,y^a  8=y+l,4^/S 

where 

(27)  /  exp  {— Xa(r  —  U)}  exp  {— X/j(^  —  t)}  dr 
Jh 

=  fexp  {—\a{t  —  <o)}  exp  {—\p{t  —  ^)}~| 

L  Xa  —  XjJ  \p  —  \a  J 

Therefore,  (26)  may  be  rewritten 

(28) 

(-l)‘-<XiXi+i---Xi-,rt  i  - ^exp  {-X.(t  -  U,)}^ - 

(X.  -  X,)  n  (X.-X,)  n  (x,  -x,) 

_  y^i,y9^ot  6—j+l,S9^p 

_l_  ^  ^  _ exp  {  Xg(f  ^)} _ _ 

(x«  _  X.)  n  (x,-x,)  n  (x«-x,)  ' 

y  =  i,y^a  S=i+l,Si>^P  _ 

In  the  first  term  inside  the  brackets,  for  each  a,  we  compute  the  sum 

r  ^  1 _ _1  exp  —  U)} 

(29)  ^  (X«-X,)[  n  (X, -X,) 

t  S=j+l,S^fi  J  y^i.y^a 

' _ 1 _ ]  exp  {— X.(t  —  fe)}  ^  exp  {— X„(i  —  ip)} 

“1  n  (X. -Xi)  |  n  (X. -X,)  n  (x. -x,) 


y  =  i,yj^a 


y=i,y^a 


since,  according  to  Lemma  1,  for  any  distinct  numbers  Xa,  X,+i,  X,+2,  •  •  •  ,  X*, 


+  •••  + 


(Xa  Xy+l)  II  (X,-+l  X7) 

y=J+2 


(x„-x*)  n  (Xk-Xy) 

7=y+i 


n  CXa-Xy) 


(30) 
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Similarly,  in  the  second  term  inside  the  brackets  of  (28)  we  find  for  each  /S, 

(31) 

r  ^  _ 1 _ 1  exp  {— X/?(^  —  ^)}  _  exp  —  to)} 

|“=*(x^-Xa)  n  (x„-x-,)|  n  (x^s-x*)  n  (x^-x,) 

L  y  =  i,y^a  J  6=j+l,5^fi  S  =  i,J^/S 

Substituting  (29)  and  (31)  in  (28)  yields 

(32)  ( -  l)‘-<X..Xi+.  •  •  •  t 

""■  n  (X.  -  X,) 

y  =  i,y 

which  is  equal  to  (23),  as  was  to  be  shown. 

3.4.  The  Polya  process.  The  Polya  process  is  determined  by  the  differential 
equation  (3)  with  the  intensity  function 


(33) 


\k(t)  = 


X  +  Xafc 
1  “f"  Xcti 


where  both  X  and  a  are  nonnegative  constants.  Solving  (3)  for  PaiU,  t),  we  have 


(34)  Pi,(to,  t) 


T(k  +  1/a) 

/I  +  Xa^oy+^z®  / 

-  1  ”1“  Xafo\ 

(k  -  i)\r(i+l/a) 

\1  +  \at/  \ 

^  1  +  \at) 

Using  this  general  formula  for  the  probabilities  P„(io,  t)  and  Pj+i,k{T,  i)  in  (10), 
we  obtain 


(35)  f  Pijito,  T)X,(r)P;+i,fc(r,  t)  dr 

Jk 

^  TU  +  l/g) _ r(^  +  l/o)  f‘  (1  +  XoioY+»'« 

u  -  i)!r(i  +  l/o)  (k-j-  l)!r(i  +  1  +  l/o)  J„  u  +  ^ar) 

(  1  +  XoloV-'  /X  +  XoA/1  +  XotV+i+i'-  /  1  +  XotX*-'-' 

V  1  +  Xot/  M  +  XotAi  +  V  1  +  W  ’ 

where  the  integral  is  simply 

(36) 

(1  +  Xa«o)*+'/“(l  +  Xai)-(*+i/“>(Xa)*-»-HX  +  Xaj)  /  (r  -  «o)'-‘(«  -  r)*-»-i  dr 

Jlo 

=  (1  +  Xa«o)*+''‘*(l  +  XaO-(*+'/“>(Xa)^-*-i 

(X  +  Xo/)((  -  (<,)»-*  ~  ~ 


^  /I  +  XafoV+^Z"  /  _  \ato  /  1\  (j  -  i)\(k  -  j  -  1)!. 

\1  “1“  Xfli  /  \  1  “1“  \at/  \  a/  (k  —  z)! 

When  the  last  expression  in  (36)  is  substituted  for  the  integral  in  (35),  the  right 
side  member  of  (35)  becomes  identical  to  that  in  (34),  proving  equality  (10)  for 
the  P61ya  process. 
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3.5.  The  pure  death  process.  The  pure  death  process  is  different  from  all  the 
preceding  processes;  in  this  case  the  population  size  is  decreasing  instead  of 
increasing  with  time  t.  The  transition  probability  PikiU,  t)  holds  for  k  ^  i  and 
satisfies  the  differential  equation 


(37) 


dt 


Pik{tOf  t)  =  —klJL{t)Pik{to,  i)  “b  +  ^)u(t)Pi,k+litQ,  t), 


fc  =  0,  1, 


where  the  intensity  function  /*(<)  is  also  known  as  the  force  of  mortality.  The 
solution  of  (37)  is 

(38)  PaCfo,  ()  =  (A((o,  ())‘(1  -  Hk,  ())<-*, 


where  h{to,  t)  denotes  exp 


k\(i-k)\ 


The  equality  for  the  death  process  as¬ 


sumes  a  form  slightly  different  from  (10) ; 

(39)  Pikito,  t)  =  f  Pijito,  T)jti{T)Pj-i,kiT,  t)  dr,  0  ^  k  <  j  ^  i. 

Ju 

Verification  of  (39)  is  straightforward.  Using  (37)  we  can  write 


(40) 


fp<> 


{k,  r)jix{j)Pj^i,k{T,  t)  dr 


= 


where  h{T,  t)  denotes  exp 
Now  let 
(41) 


k\ij  -l-k) 
j  m(?)  and  h{to,  r)  denotes  exp 


ar  \  -  1  —  h{k,  t)  .  ,  .  _  h{to,  t)u{t)  dr 

°\r)  —  1  I./#  A* 


1  —  h{to,  t) 


1  -  h{to,  t) 


Then  (40)  may  be  rewritten  as 
(42) 

*))‘(1  -  hito,  or--  -  9(r)).— ‘  <M(r) 

=  k\(/-  k)\  ~ 

=  Pikito,  t), 

which  proves  the  equality  in  (39). 
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4.  An  application 

Theoretical  significance  of  equality  (10)  is  yet  to  be  assessed,  depending  to  a 
great  extent  on  its  relevance  in  developing  stochastic  processes.  The  following 
example  illustrates  its  usefulness  as  a  means  of  deriving  certain  transition 
probabilities. 

The  example  in  question  is  the  so-called  “simple  stochastic  epidemic,”  which 
has  been  extensively  studied  by  N.  T.  J.  Bailey  [1].  According  to  this  model,  a 
population  consists  of  two  groups  of  individuals:  infectives  and  susceptibles.  At 
the  initial  time  U,  there  are  one  infective  and  N  susceptibles.  For  each  t,  for 
t  G  [<o,  ),  there  are  X{t)  infectives  and  N  -{•  1  —  X{t)  susceptibles.  The  proba¬ 

bility  distribution  of  the  random  variable  X{t)  satisfies  the  system  of  differential 
equations 

(43)  ~  —O'k0(t)Pve(to,  t)  +  a*_u8(i)Pi,t_i(fo,  t), 

k=l,  ,N+1, 

where 

(44)  fli  =  k{N  1  —  k)  fc  =  1,  •  •  •  ,  AT;  oo  =  =  0, 

with  the  initial  condition  PniU,  U)  =  1,  and  the  constant  ^(t)  is  known  as  the 
infection  rate.  The  Laplace  transform  and  the  probability  generating  function 
have  been  used  to  solve  the  differential  equations  (43).  Because  the  coefficient 
ak  is  a  quadratic  function  of  k,  for  each  k,  the  computations  involved  are  quite 
complex.  The  partial  differential  equation  for  the  probability  generating  func¬ 
tion,  for  example,  is  of  the  second  order.  However,  when  equality  (10)  is  used, 
one  can  write  down  the  solution  almost  immediately.  The  complete  solution  and 
the  detailed  discussion  on  related  points  have  been  given  in  [7],  and  will  not  be 
repeated  here. 

6.  The  equalities  in  a  two  dimensional  process 

Equality  (10)  can  be  extended  to  multidimensional  processes,  where  the  ran¬ 
dom  variables  concerned  are  the  number  of  transitions  rather  than  population 
sizes.  To  be  specific,  we  use  a  two  dimensional  process  for  illustration.  In  the 
birth,  death  and  other  processes  discussed  in  Section  3,  there  is  only  one  transient 
state  for  each  individual:  the  state  of  “living.”  In  a  two  dimensional  process, 
each  individual  may  be  in  either  one  of  two  transient  states,  /Si  and  S2.  State  Si 
may  be  interpreted  as  the  healthy  state,  and  Sz  the  illness  state.  In  addition, 
there  is  an  absorbing  state  R,  the  death  state.  A  person  is  in  state  <Si  if  he  is 
well,  in  Sz  if  he  is  ill.  A  transition  from  Si  to  Sz  means  the  occurrence  of  an 
illness  or  a  relapse,  while  a  transition  from  Sz  to  Si  means  recovery.  When  a 
person  dies,  he  enters  the  death  state  R  from  either  /Si  or  Sz  depending  upon 
whether  he  is  in  state  Si  or  Sz  at  the  time  of  death.  Fix  and  Neyman  have  dis¬ 
cussed  extensively  this  model  in  their  study  of  the  probabilities  of  relapse,  re¬ 
covery,  and  death  for  cancer  patients  [6]  (see  also  Du  Pasquier  [4]). 
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During  a  time  interval  (to,  t)  an  individual  may  leave  one  transient  state  for 
another.  We  are  interested  in  the  number  of  transitions  that  he  makes  between 
the  two  transient  states  and  the  corresponding  probabilities,  which  are  defined 
as  follows: 


(45)  P^\to,  t)  =  Pr  {an  individual  in  state  Sa  at  time  to  will  leave  Sa  m  times 
during  (io,  0  and  will  be  in  at  time 
=  Pr{Ma^(to,  t)  =  m},  a,  =  1,  2;  m  =  0,  1,  •  •  •  . 

The  random  variable  Mafi(to,  t),  corresponding  to  the  probability  P^\to,t),  is 
thus  the  number  of  times  the  individual  leaves  Sa  for  Sp  before  he  reaches  Sp  at 
time  t.  The  sums 


E  t)  =  Paa(to,  t), 

(46) 

t  PS’(<0,  t)  =  t), 

m-1 

are  the  probabilities  that  the  individual  will  be  at  time  t  in  Sa  and  Sp,  respectively, 
regardless  of  the  number  of  transitions  he  makes  between  U  and  L  We  assume 
that,  for  each  t,  for  io  ^  t  ^  t,  the  derivatives 


(47) 


f-^P««(r,<) 


Vap 


exist  and  are  independent  of  time  t,  so  that  Vaa  is  a  negative  constant  and  a 
positive  constant.  Explicit  formulas  of  the  probabilities  P^\to,  t)  have  been 
derived  in  terms  of  Vaa  and  Pap  (see  [2],  Chapter  5). 

Corresponding  to  equality  (10)  there  are  four  equalities  in  the  two  dimensional 
process.  They  are 


(48) 

P<3>((o,  0  = 

f  P&ih,  ()  dr, 

(49) 

PS\u,  0  = 

1  t)v,.PS-*(t,  t)  dr, 

JtQ 

(50) 

<)  = 

r  t)»s.P2:-'"(t,  <)  dr, 

and 

(51) 

PS’((o,  0  = 

j  P^{U,r)r„lPSi~^(r,t)dr, 

Ju 

for  any  fixed  but  arbitrary  j,  for  j  between  0  and  m.  Equality  (48),  for  example, 
holds  for  j  =  0,  •  •  •  ,  m  —  1;  w’^hile  (49)  holds  foij  =  1,  •  •  •  ,  w  —  1.  These  four 
equalities  can  be  verified  in  a  similar  manner  as  equality  (10).  For  the  verifica¬ 
tion  of  the  equality  in  (48),  consider  that  an  individual  in  state  Sa  at  time  t^ 
will  leave  S«  m  times  and  be  in  Sp  at  time  t,  and  let  j  be  a  fixed  number  with 
0  ^  j  <  m.  Let  the  (j  +  l)th  exit  transition  from  S«  to  Sp  take  place  in 
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(r,  T  +  dr),  SO  that  at  r  he  is  in  state  Sa,  the  remaining  m  —  j  —  1  transitions 
from  Sp  occurring  during  (t,  t) ;  the  probability  of  this  sequence  of  events  is 

(52)  P^JUU,  dT]P^-'-'^(r,  <)• 

Integrating  (52)  from  t  =  Jo  to  t  =  f  gives  the  equality  in  (48). 

It  has  been  shown  in  [2],  pp.  102  to  104,  that  the  multiple  transition  proba¬ 
bilities  P^\to,  t)  and  P^\to,  t)  satisfy  the  equalities  (48)  through  (51)  for  every 
m.  Two  simple  cases  are  given  below. 

It  is  obvious  that  for  m  =  0, 

(53)  P^{to,  t)  =  exp  {vaa{t  —  to)},  P^p  (U,  t)  =  BXp  —  «o)}  • 

For  m  =  1,  Pafiito,  t)  is  the  probability  that  exactly  one  transition  from  Sa  to  Sp 
occurs  in  (to,  t),  and  P^ito,  t)  is  the  first  return  probability  to  the  original  state 
Sa  at  t  after  having  left  Sa  once.  The  first  passage  probability  is  given  by 

(54)  t)  =  — — —  (exp  {vaa{t  -  to)}  —  exp  —  U)}),  a  9^ 

Vaa  —  V/5/J 

According  to  equality  (48),  we  have 

(55)  P^c${to,  t)  =  \  Pr  («o,  r)va&P^^^{r,  t)  dr, 

Ju 

or,  substituting  (53)  and  (54)  in  (55), 


(56) 


Vgp 

Vaa  —  V0^ 


(exp  {Vaa{t  -  to))  -  exp  {v^fiit  -  <o)} ) 


exp  {Paair  -  to)}vap  exp  {ppp{t  —  t)}  dr, 


which  is  easily  shown  to  be  true. 

Equalities  (48)  and  (50)  (or  (49)  and  (51))  can  also  be  used  to  derive  the  gen¬ 
eral  formulas  for  the  probabilities  P^a\to,  t)  and  P%^{to,  t).  The  probability 
Papito,  t),  if  it  is  unknown,  can  be  obtained  from  (55).  Using  the  known  proba¬ 
bility  Pa0\to,  t),  we  can  derive  the  first  return  probability  Paaih,  t)  from  equality 
(50)  for  m  =  I,  j  =  I, 

(57)  P^^ito,  t)  =  I  PL^ito,  T)ppaP^^a{T,  t)  dr. 

J  to 

Substituting  (56)  for  P^^^{to,  r)  and  (53)  for  Pi'^air,  t)  in  (57)  and  integrating  the 
resulting  expression  yields 


(58)  PiLH^O,  t)  =  -  to)  exp  {Paa{t  -  to)} 

\Vaa  V0^) 

-  (exp  {Paa{t  —  to)}  -  exp  {p^0{t  -  <o)})]. 

Now  using  equalities  (48)  and  (50)  successively  for  w  =  2,  we  obtain  the  second 
passage  probability 
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(59) 

Pap  {to,  t)  =  [("««  “  m){t  -  ^)(exp  {vaa{t  “  to)}  +  exp  {vpp{t  —  to)}) 

—  2(exp  {vaa{t  —  to)}  —  exp  {vpp{t  —  <o)})] 

and  the  second  return  probability 

2  2 

(60)  P'S.iU,,  1)  =  [K-'..  -  mm  -  u.y  exp  K.(i  -  <c,)} 

—  {Vaa  —  Vpp){t  -  to)  {2  exp  {vact{t  “  U)}  +  exp  {ppp{t  —  fo)}) 
+  3 (exp  {Paa{t  —  to)}  -  exp  {ppp{t  -  «o)})]. 

The  probabilities  P^\to,  t)  and  P^^{to,  t)  can  all  be  successively  derived  in  the 
same  manner  beginning  with  m  =  3.  It  is  interesting  to  note  that,  using  this 
approach,  the  multiple  transition  probabilities  can  be  derived  even  when  the 
intensity  functions  Paa{t)  and  Pap{t)  are  functions  of  time. 
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